Enumerative encoding/decoding of variable-to-fixed-length codes for memoryless sources

We offer novel algorithms for efficient encoding/decoding of variable-to-fixed length codes, requiring at most quadratic amount of space: O(L), where L is the depth of a coding tree. This is a major improvement compared to exponential O(2) usage of space by conventional techniques using complete representations of coding trees in computer’s memory. These savings are achieved by utilizing algebraic properties of VF coding trees constructed by using Tunstall or Khodak algorithms, and employing combinatorial enumeration techniques for encoding/decoding of codewords. The encoding/decoding complexity of our algorithms is linear with the number of symbols they process. As a side product, we also derive an exact formulae for the average redundancy of such codes under memoryless sources, and show its usefulness for analysis and design of codes with small number of codewords. 1 Definitions Consider a memoryless source S producing symbols from an input alphabet A = {a1, . . . , am} (2 6 m < ∞) with probabilities {pi = P (ai), i = 1, . . . , m}. By pmin, pmax we denote the probabilities of the leastand the most-likely symbols correspondingly, and assume that 0 < pmin 6 pmax < 1. Assume that a message Σ = {aik}k=1 is an infinite-length sequence of symbols produced by the source S. The main idea of variable length coding is to define a set of words X = {xj ∈ A∗, j = 1, . . . , M} (m 6 M < ∞), such that any message Σ can be uniquely represented by a sequence of words from X, and then map these words xj into codewords φ(xj) formed from letters of an output alphabet B = {b1, . . . , bn} (2 6 n < ∞): {aik}k=1 = {xjr}r=1 → {φ (xjr)}r=1 = {bis}s=1 . We further assume that the mapping φ is injective, and that the output code {φ (xj)} is uniquely decodable [4]. In this paper we only consider coding systems producing codewords φ(xj) with the same length. For example, we can simply pick |φ(xj)| = dlogn Me (1 6 j 6 M), and use indices of words xj to produce their codes. Such coding systems are called variable-length-to-block (VB) or variable-to-fixed-length (VF) codes.